Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove mkl in for MacOS builds #3896

Merged
merged 3 commits into from
Mar 16, 2023
Merged

Remove mkl in for MacOS builds #3896

merged 3 commits into from
Mar 16, 2023

Conversation

atalman
Copy link
Contributor

@atalman atalman commented Mar 16, 2023 •
edited
Loading

Remove mkl constraint to fix MacOS builds

This is used to fail, here is an example: https://github.com/pytorch/test-infra/actions/runs/4431671908/jobs/7774880091

@vercel
Copy link

vercel bot commented Mar 16, 2023

@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 16, 2023
malfet
malfet reviewed Mar 16, 2023
View reviewed changes
tools/pkg-helpers/pytorch_pkg_helpers/macos.py Outdated
@@ -6,6 +6,6 @@ def get_macos_variables(arch_name: str, python_version: str = "3.8") -> list:
]

if arch_name != "arm64" and python_version != "3.11":
variables.append("export CONDA_EXTRA_BUILD_CONSTRAINT='- mkl<=2021.2.0'")
variables.append("export CONDA_EXTRA_BUILD_CONSTRAINT='- mkl<=2023.0.0'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this constraint is there in the first place?

Copy link
Contributor Author

@atalman atalman Mar 16, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done removed MKL constraint

@atalman atalman changed the title Advance mkl in for MacOS builds Remove mkl in for MacOS builds Mar 16, 2023
malfet
malfet approved these changes Mar 16, 2023
View reviewed changes
Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check that it does not introduce changes in binary dependencies (i.e. this change originally was in place to prevent spurious dependency on libmkl_*.so.1 instead of libmkl_*.so

@atalman
Copy link
Contributor Author

atalman commented Mar 16, 2023

Please check that it does not introduce changes in binary dependencies (i.e. this change originally was in place to prevent spurious dependency on libmkl_*.so.1 instead of libmkl_*.so

Sounds good will run ldd test on generated binaries

@atalman atalman merged commit 4f970b4 into pytorch:main Mar 16, 2023
@atalman
Copy link
Contributor Author

atalman commented Mar 17, 2023 •
edited
Loading

running otool with this fix on MacOS x86 I see following:

@rpath/libtorchaudio.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_ilp64.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_core.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_thread.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
	@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

With python 3.9 MacOS x86 without this fix:

otool -L libtorchaudio.so 
libtorchaudio.so:
	@rpath/libtorchaudio.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

Current prod build 3.10 MacOS x86 without this fix using conda-forge:

otool -L libtorchaudio.so 
libtorchaudio.so:
	@rpath/libtorchaudio.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

Running python 3.10 release MacOS Arm64 :

 otool -L libtorchaudio.so 
libtorchaudio.so:
	@rpath/libtorchaudio.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.23.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)

All smoke test are passing however, and everything looks green
cc @malfet

@atalman
Copy link
Contributor Author

atalman commented Mar 17, 2023 •
edited
Loading

to followup on this, bumping constraint version also produces these dependencies using otool:

	@rpath/libtorchaudio.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtorch_cpu.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libc10.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_ilp64.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_core.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libmkl_intel_thread.1.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
	@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

Taken from this job:
https://github.com/pytorch/test-infra/actions/runs/4441037568/jobs/7795607186#step:7:37

@atalman atalman mentioned this pull request Mar 20, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@malfet malfet malfet approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@atalman @malfet @facebook-github-bot